System for managing display and retrieval of image content on a network with image identification and linking to network content

ABSTRACT

A system manages display and retrieval of image content on a network by identifying the image and linking the image to related information, such as licensing information or usage rights. The system manages the display of image content stored within a network by associating thumbnail images that link to versions of the image content stored on a network. One example is a thumbnail that acts as a bookmark linking to image signal content stored on a distributed network of computers, such as links to web pages accessible on the internet. Corresponding methods are also provided.

RELATED APPLICATION

This application is a continuation of application Ser. No. 11/050,407,filed Feb. 2, 2005 (U.S. Pat. No. 7,685,426), which is a continuation inpart of application Ser. No. 10/112,884, filed Mar. 29, 2002 (publishedas US 2002-0196272 A1), which is a continuation of application Ser. No.09/165,142, filed Oct. 1, 1998 (U.S. Pat. No. 6,421,070). This patentapplication is also related to: application Ser. No. 08/746,613, filedNov. 12, 1996 (U.S. Pat. No. 6,122,403); application Ser. No.09/972,792, filed Oct. 5, 2001 (published as US 2005-0058313 A1);application Ser. No. 09/612,177, filed Jul. 6, 2000 (U.S. Pat. No.6,681,029); application Ser. No. 08/746,613, filed Nov. 12, 1996 (U.S.Pat. No. 6,122,403); application Ser. No. 08/649,419, filed May 16, 1996(U.S. Pat. No. 5,862,260); and PCT/US96/06618, filed May 7, 1996(published as WO 96/36163). The above patents and patent applicationsare hereby incorporated by reference.

FIELD OF THE INVENTION

The invention relates to managing, identifying, indexing, and linking tocontent, and particularly image signals, on networks, such as web pageson the Internet.

BACKGROUND AND SUMMARY OF THE INVENTION

There are numerous commercially available programs called “browsers”that facilitate accessing and displaying data. The two leadingcommercially available browsers are the “Netscape Communicator” which isdistributed by Netscape Corporation of Mountain View, Calif. and the“Internet Explorer” browser that is distributed by Microsoft corporationof Redmond, Wash.

Browsers allow one to utilize the internet to access web pages locatedat remote sites. A browser displays web pages in a window on a displaydevice. The web pages that are displayed can contain both text andimages.

Technology called stegangraphy had been developed which allows one tostore digital data in an image. Such data is frequently termed a“digital watermark”. The digital data is not visible when an imagecontaining such data is displayed with a conventional browser; however,the image can be passed through a special program which can detect andread the hidden data. Systems for storing digital data in images and forthen reading such data from the images are for example shown in U.S.Pat. No. 5,636,292 and in U.S. Pat. No. 5,748,783. Such technology isalso discussed in the “Communications of the ACM” published July 1998Vol. 41. No. 7 pages 31 to 77.

The following disclosure describes programs that work with a browser toprovide functions that are not performed by prior programs. A typicalweb page displayed by a browser contains several images. With aconventional browser, a user can not visually determine if any of theimages displayed on a web page contain a watermark, and cannot createand use image bookmarks.

One method described in this disclosure is a method of managing content,and in particular, managing content on the Internet. The methodretrieves a web page that includes an image and detects whether theimage included within the web page is embedded with a digital watermark.It generates an indicia associated with an image included in the webpage that is embedded with a digital watermark. The indicia indicate tothe user which images include watermarks. The watermarks may be used toconvey links to related web pages or specific information about theimages, such as usage rights and licensing information. Variations ofthis method create image bookmarks to web pages including images usingthumbnails of those images.

Another aspect of the disclosure is a system for managing content. Thesystem comprises a first program for retrieving web pages includingimages. It also includes a second program for extracting an image from aweb page, creating a thumbnail of the image, and forming an imagebookmark linking the thumbnail to the web page that the image has beenextracted from. The thumbnails are used to create a visual index tocorresponding web pages from which the images originated on theInternet, for example.

Another aspect of the disclosure is a method of visual indexing ofcontent on a network, such as the Internet. The method retrieves a webpage, extracts an image included on the web page, generates a thumbnailof the image, and creates a link between the thumbnail and a location ofthe web page from which the image has been extracted.

One embodiment is implemented as an adjunct to a convention browserwhich displays web pages including images. In this embodiment, visualindicia is placed on images that include a digital watermark so that auser can know that the image contains hidden watermark data. By clickingon the indicia which is placed on the image, the user will be linked tothe web page identified by the watermark data hidden in the image. Inthis embodiment, such a link can be established without the web pagedesigner having to include a tag in the web page which displays theoriginal image. This embodiment also opens a separate window depicting athumbnail of each image in a web page. If a user right clicks on one ofthe thumbnails in this window the image will be added to a list ofimages in a special image bookmark file. When a user opens the bookmarkweb page, thumbnails of all the stored images in the bookmark file aredisplayed. A user can recall the web page which originally contained theimage by clicking on the thumbnail.

Another aspect of the disclosure is a system comprising: a memory; and aprocessor in communication with the memory and programmed to retrieve animage signal. The processor is programmed to compute identifyinginformation from the image signal, and to associate the identifyinginformation with a link to additional information related to the imagesignal. The processor is also programmed to create a thumbnail of theimage signal, and programmed to form an image bookmark linking thethumbnail to a network location where the image signal is stored.

Yet another aspect of the disclosure is a method comprising: retrievingan image signal; using a programmed processor, computing identifyinginformation from the image signal; associating the identifyinginformation with a link to additional information related to the imagesignal; creating a thumbnail of the image signal; forming an imagebookmark linking the thumbnail to a network location where the imagesignal is stored.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram of a browser.

FIG. 2 is a block diagram of an embodiment.

FIG. 3 is flow diagram of the operation of the preferred embodiment.

FIG. 4 is a diagram of a display screen of the preferred embodiment.

FIG. 5 is a diagram of a display screen showing bookmarks.

FIG. 6 is a flow diagram of the operation of the bookmark program.

DETAILED DESCRIPTION

An embodiment is implemented utilizing what is known in the art as“helper application” for the Internet Explorer 4.0 browser. The InternetExplorer 4.0 browser is commercially distributed by MicrosoftCorporation. FIG. 1 is a block diagram of the commercially availableInternet Explorer 4.0 browser 10. Browser 10 provides a mechanism forreceiving and displaying data (called Web Pages) received from the WorldWide Web (which is often referred to as W W W).

The embodiment described herein is designed to operate based onwatermarks which have a particular format designed by DigimarcCorporation. Many of the commercially available programs which caninsert watermarks in images and which can detect watermarks in imagesutilize this watermark format. For example, the Digimarc format is usedby the following commercially available programs: “Adobe PhotoShop”Versions 4.0 and 5.0 and “Adobe ImageReady” Version 1.0 which aremarketed by Adobe Corporation, “CorelDRAW” Versions 7 and 8, and “CorelPHOTO-PAINT” Versions 7 and 8 which are marketed by Corel Corporation,and Micrografx Webtricity” Versions 1 and 2, “Micrografx Graphics Suite2”, and “Micrografx Picture Publisher” Versions 7 and 8 which aremarketed by Micrografx Corporation.

The base program in the Internet Explorer 4.0 browser, that is, theprogram which begins the operation of the browser 10 is IEXPLORE.EXE 11which is shown in FIG. 1. Program 11 calls the web browser controldynamic link library SHDOCVW.DLL 12. As stated in the documentation ofthe Internet Explorer provided by Microsoft SHDOCVW.DLL 12 “supplies thefunctionality associated with navigation, in-place linking, favoritesand PICS support.” SHDOCVW.DLL 12 in turn hosts or calls the MSHTML.DLL13 dynamic link library. MSHTML.DLL “performs the HTML parsing andrendering” and also “exposes the HTML document through the Dynamic HTMLObject Model” 14. The HTML Object Model 14 hosts the Active X Control14A, the Active X Engine 14B, the JAVA VM 14C and the Plug In Applet14D. The various components in browser 10 store and retrieve informationfrom URL Cache 15. The operation and function of the various componentsof the Internet Explorer browser are described in the publicly availableliterature (and on the web site) provided by Microsoft Corporation.

An embodiment adds a browser helper object 21 and two other programs 22and 23 as shown in FIG. 2. Program 22 is a conventional program designedto detect a watermark in an image. One example of such a program isshown in U.S. Pat. No. 5,636,292. Another program for detectingwatermarks is shown in U.S. Pat. No. 5,689,587. The browser helperobject 21 interfaces with the dynamic link library MSHTML.DLL 13 andwith the URL cache 15.

FIG. 4 illustrates an example of a web page being displayed in a window40 on display screen 41 by browser 10. The example shown in FIG. 4 hasfour images designated image 1, image 2, image 3 and image 4. It shouldunderstood that the number of images and the placement of the imagesvaries in each web page displayed and it is determined by the person whocreates a web page. Furthermore, text may be interspersed with theimages. The example shown in FIG. 4 is a simple example of a web pagewhich is herein used to illustrate the operation of an embodiment.

The embodiment detects which images on a web page contain a watermark.The images which contain watermarks are flagged or noted by means of anindicia which is added to the lower right hand corner of any images thatcontain watermarks. In the example shown in FIG. 4, image 3 contains awatermark and thus indicia 46 appears on the lower right hand corner ofthe image. Indicia 46 could for example be a logo which identifies aparticular company or it could be any other easily identified mark orsymbol. It could be as simple as the letters WM or it could be a smallmulticolored image. The indicia which is displayed is stored in GIF file(Graphic Interchange Format file) and referenced by HTML code (HyperTextMarkup Language code) which causes the indicia to be displayed.

The data in the watermarks (which have the previously defined format)includes a particular HTML address. In the embodiment, if a user clickson the indicia 46, a link is created and executed to a particular webpage on a particular server herein identified as“www.digimarc.com/cgi-bin”.

The program 20 also opens a separate window 42 and it places a thumbnail(i.e. a reduced version) of each image in window 42. In the exampleshown in FIG. 4, thumbnails 42A, 42B, 42C and 42D are small versions ofimages 1 to 4 respectfully.

Program 20 also creates an image or “button” 45 which contains thesymbol BM. If a user right clicks on one of the images in window 42,that image is saved as a bookmark. If at a later time a user clicks onthe BM image 45, the system displays a list of the previously savedimages 53 a to 53 h as shown in FIG. 5. When a user clicks on one of thedisplayed images 53 a to 53 h, a link is executed to the page from whichthe image originated, and thus that page is again displayed by thebrowser 10.

FIG. 3 is a block flow diagram of the operation of the browser helperobject 21 and program 22. Block 31 indicates that initially the browser10 receives data and it renders images in window 40 on the screen 41 asis normal for the operation of the browser 10. Block 32 indicates thatwhen the download operation is complete, MSHTML.DLL 13 sends a “Downloadcomplete event” and a “Document Object” to Browser helper Object 21. TheDocument Object includes the URL addresses of each of the images in thepage that is displayed. The characteristics of a Download Complete Eventand a Document Object is explained in the documentation provided byMicrosoft Corporation. Block 34 indicates that Browser Helper 21 sends arequest to MSHTML for the address in cache 15 of one of the URLaddresses which it previously received. The documentation supplied byMicrosoft corporation explains how the above operations are performed.

Block 35 indicates that when browser helper 21 receives a DownloadCompete Event from MSHTML.DLL 13, the browser helper 21 queries the“Document Object”. The images which are in the page being displayed areavailable to the browser helper 21 in the current “Document”. Thebrowser helper 21 retrieves the image data from the URL cache 15 andprocesses it as follows: The image is passed through watermark detectorprogram 22 to determine if the image contains a watermark and bookmarkprogram 23 places a thumbnail of the image in window 32.

As indicated by block 38, if no watermark is detected (and if this isnot the last image which appears in the window being displayed) theprogram flow returns to block 34 and the process repeats for anotherimage. If the image being processed is the last image in a window theprocess stops and does not begin again until browser helper 21 receivesanother “download Compete Event” signal.

If a watermark is detected by watermark detector 22, the processproceeds to block 39. As indicated by block 39, in this event helperProgram 21 calculates the position of the lower right hand corner of theimage and “inserts” a HREF and an IMG tag to the current document objectof MSHTML.DLL. In response to the HREF and IMG tag, MSHTML.DLL willdisplay an indicia such as indicia 46 in the lower right hand corner ofthe image with the watermark. The browser helper can calculate the lowerright hand corner of the image where the indicia 46 is to be insertedfrom the location data of the image. The location where the image is tobe inserted is given in the HREF command. The following is an example ofand HREF and IMG tags:

<a href=”http://www.digimarc.com/cgi-bin/c1.p1?4+404407+0.0.+1.” <IMGSRC=’ind46.gif’ STYLE=”position:absolute;left:125;top:200;filter;alpha(opacity)”> </a>

-   -   where: ind46.gif is a GIF image of the indicia 46.

The general format of such HREF and IMG tags is conventional. Likewisethe technique for “inserting” a HREF and an IMG tag to the currentdocument object of MSHTML.DLL is known.

When watermark detector 22 determines that a particular image contains awatermark, the browser helper 21 inserts tags such as the above to theMSHTML.DLL 13 which then superimposes indicia 46 over the image beingdisplayed by the browser 10.

An Active X browser control program publicly available from MicrosoftCorporation is used to display the thumbnails in window 42 as shown inFIG. 4. The Active X browser control program is also used to display theimage bookmarks as shown in FIG. 5. A specific example of image HTMLused to display thumbnails in window 42 using an Active X browsercontrol is given below.

<table width= 72> <tr><ahref=‘http://safari.altavista.digital.com/cgi-bin/VP?START=1’title=‘http://safari.altavista.digital.com/cgi-bin/VP?START=1’><fontsize=1 color=‘blue’ face=‘Verdana,Arial,Helvetica’> <Ponmouseover=“this.style.color = ‘red’”; onmouseout= “this.style.color =‘blue’”;>Corbis Picture Experience - AltaVista - BETA TEST<P></font></a></tr> </table> <table cellspacing=‘1’ cellpadding=‘1’,border=‘0’ width=‘72’> <tr>   <td rowspan=‘2’ ><ahref=‘http://safari.altavista.digital.com/cgi-bin/VP?START=1‘><imgsrc=“http://safari.altavista.digital.com/Assets/coravcpe.gif” border=‘0’title=‘http://safari.altavista.digital.com/Assets/coravcpe.gif’border=‘0’ height=‘30’ width=‘70’></a> </td> {close oversize bracket}    <td></td> </tr> <tr>     <td></td> </tr> <tr>   <td rowspan=‘2’ ><ahref=‘http://safari.altavista.digital.com/cgi-bin/VP?START=1’><imgsrc=“http://safari.altavista.digital.com/Assets/coravtitle.gif”border=‘0’title=‘http://safari.altavista.digital.com/Assets/coravtitle.gif’border=‘0’ height=‘50’ width=‘70’></a> </td> {close oversize bracket}    <td></td> </tr> <tr>     <td></td> </tr> <tr>   <td rowspan=‘2’ ><ahref=‘http://safari.altavista.digital.com/cgi-bin/VP?START=1’><imgsrc=“http://safari.altavista.digital.com/Assets/corav123.gif” border=‘0’title=‘http://safari.altavista.digital.com/Assets/corav123.gif’border=‘0’ height=‘52’ width=‘37’></a> </td> {close oversize bracket}    <td></td> </tr> <tr>     <td></td> </tr> <tr> </table>

The above example includes three images. The brackets at the right handside of the above code indicate the sections of code which display eachimage. When in use, the number of images equals the number of images ona particular web page which is being displayed.

A block diagram of the program used to add images to the list of imagebookmarks is shown in FIG. 6. With reference to FIG. 4, if a user wouldlike to add one of the images 1 to 4 to a list of images that the userhas bookmarked, the user would right clicks on the thumbnail of thatimage which shown in window 42 (block 61). As a result of the rightclick, a drop down window (not specifically shown) will appear askingfor the name the user wants to associate with the image (block 62). Itis noted that the use of drop down windows to add information isconventional and well known. When the drop down window appears, the userenters a name and the image is added to the list of bookmarks with thename entered.

Similar HTML code to that given above is used with the Active X Browsercontrol to display bookmarks as shown in FIG. 5. The following is anexample of HTML code which displays bookmarks as shown in in FIG. 5.Such HTML is conventional and many alternative sequences of HTML can beused to generate a similar page of images.

<html> <head> <title>Digimarc Watermark Explorer Bookmark</title> <bodyvlink=‘blue’ link=‘blue’> <table width= 100 align=‘center’> <tr> <td><ahref=‘http://www.cnn.com/’ name=‘Bookmark_C’><font size=1 color=‘blue’face=‘Verdana,Arial,Helvetica’><U onmouseover=“this.style.color= ‘red’”;onmouseout=“this.style.color = ‘blue’”; >CNN Interactive</U></font></a></td></tr> {close oversize bracket} <tr> <td width=‘100’align=‘center’> <a href=‘http://www.cnn.com/’><imgsrc=“http://www.cnn.com/images/1998/05/homepage/cnnin.logo.gif”alt=‘http://www.cnn.com/’ border=‘0’ height=‘52’ width=‘70’></a> </td></tr> </table> <table width= 100 align=‘center’> <tr> <td><a href=“name=‘Bookmark_M’><font size=1 color=‘blue’face=‘Verdana,Arial,Helvetica’><U onmouseover= “this.style.color =‘red’”; onmouseout=“this.style.color = ‘blue’”; >Microsoft Inyestor</U></font></a></td></tr> <tr> <td width=‘100’ align=‘center’> <ahref=”><img src=“http://investor.msn.com/common/images/invlogo.gif”alt=“ border=‘0’ height=‘48’ width=‘70’></a> </td> </tr> </table> <tablewidth= 100 align=‘center’> <tr> <td><a href=“ name=‘Bookmark_P’><fontsize=1 color=‘blue’ face=‘Verdana,Arial,Helvetica’><Uonmouseover=“this.style.color = ‘red’”; onmouseout=“this.style.color =‘blue’”; >PC World</U> </font></a></td></tr> <tr> {close oversizebracket} <td width=‘100’ align=‘center’> <a href=”><imgsrc=“http://www.idg.net/channels/ie4/english/images/idg_logo.gif” alt=“border=‘0’ height=‘52’ width=‘70’></a> </td> </tr> </table> </body></html>

The brackets to the right of the above HTML show the sections of HTMLused to display one bookmark. Naturally it should be understood that theabove is merely an example of one particular set of HTML that can beused to display a list of bookmarks. Various other sequences of code canbe used to obtain similar functions. HTML code such as that shown aboveis conventional and widely used.

Another feature that can be added to the preferred embodiment is thatwindow 42 can be used to display a visual history of the pages that havebeen viewed. That is one could include in window 42 a button that wouldallow one to scroll back through the thumbnails previously displayed inthe window.

While the invention has been showed with respect to a preferredembodiment thereof, it should be understood that various changes andmodifications can be made without departing from the spirit of theinvention. The scope of applicant's invention is defined by thefollowing claims.

What is claimed is:
 1. A system comprising: a memory; and a processor incommunication with the memory configured to: retrieve an image signal;compute identifying information from the image signal; associate theidentifying information with a link to additional information related tothe image signal; create a thumbnail of the image signal; and form animage bookmark linking the thumbnail to a network location where theimage signal is stored.
 2. The system of claim 1, wherein the processoris configured to detect a digital watermark in the image signal, whereinthe digital watermark provides the identifying information linking theimage signal to additional information on a network.
 3. The system ofclaim 1, wherein the processor is configured to generate graphicalindicia indicating that the image signal is linked to the additionalinformation.
 4. The system of claim 3, wherein the additionalinformation comprises a web page.
 5. The system of claim 4, wherein theweb page provides usage rights for the image signal.
 6. The system ofclaim 4, wherein the web page provides licensing information for theimage signal.
 7. The system of claim 1, wherein the processor isconfigured to generate indicia for an image signal in which a digitalwatermark is detected, wherein the indicia serves to identify that theimage signal includes a digital watermark.
 8. The system of claim 1,wherein the processor is configured to extract the image signal from aweb page.
 9. The system of claim 8, wherein the processor is configuredto provide a user interface including a display of a web page in a firstwindow and a thumbnail of an image signal in the web page in a secondwindow.
 10. The system of claim 9, wherein the thumbnail of the imagesignal is annotated with indicia when a digital watermark is detected inthe image.
 11. The system of claim 1, wherein the processor isconfigured to maintain and display a list of thumbnails as imagebookmarks linking the thumbnails to corresponding image signals atnetwork locations where the image signals are stored.
 12. A methodcomprising: retrieving an image signal; computing, using a processor,identifying information from the image signal by detecting a digitalwatermark in the image signal, wherein the digital watermark providesthe identifying information; associating the identifying informationwith a link to additional information related to the image signal;creating a thumbnail of the image signal; and forming an image bookmarklinking the thumbnail to a network location where the image signal isstored.
 13. The method of claim 12, wherein computing identifyinginformation comprises detecting a digital watermark in the image signal,wherein the digital watermark provides the identifying informationlinking the image signal to additional information on a network.
 14. Themethod of claim 12, further comprising generating graphical indiciaindicating that the image signal is linked to the additionalinformation.
 15. The method of claim 14, wherein the additionalinformation comprises a web page.
 16. The method of claim 15, whereinthe web page provides usage rights for the image signal.
 17. The methodof claim 15, wherein the web page provides licensing information for theimage signal.
 18. The method of claim 12, further comprising generatingindicia for an image signal in which a digital watermark is detected,wherein the indicia identifies that the image signal includes a digitalwatermark.
 19. The method of claim 12, further comprising extracting theimage signal from a web page.
 20. The method of claim 19, furthercomprising providing a user interface including a display of a web pagein a first window and a thumbnail of an image signal in the web page ina second window.
 21. The method of claim 20, wherein the thumbnail ofthe image signal is annotated with indicia when a digital watermark isdetected in the image.
 22. The method of claim 12, further comprisingmaintaining and displaying a list of thumbnails as image bookmarkslinking the thumbnails to corresponding image signals at networklocations where the image signals are stored.
 23. A non-transitorycomputer-readable medium having instructions stored thereon, theinstructions comprising: instructions to retrieve an image signal;instructions to compute identifying information from the image signal bydetecting a digital watermark in the image signal, wherein the digitalwatermark provides the identifying information; instructions toassociate the identifying information with a link to additionalinformation related to the image signal; instructions to create athumbnail of the image signal; and instructions to form an imagebookmark linking the thumbnail to a network location where the imagesignal is stored.